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ABSTRACT $•- ':v •. ■ ■' &/; 

‘ A basic description of the Rasch model, and a brief 

review of the work previously done on the model is related* Simulated 
data was used to test goodness of fit to the Rasch model* It was 
found that data with near perfect fit to the Guttman model provides 
/ perfect fit to the Rasch model, though a perfect Guttman scale 
cullapses. Random data also provides a good fit* The effects of 
varying the standard deviations on normally distributed total scores 
and the ranges on uniformly distributed scores and the effects of 
combining two sets of independent data are analyzed in some detail. 
[Not available in hard copy due to marginal legibility of original 
document. ] (DG) 
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• Much has been said about the characteristics of data 
that affect the various statistics used in evaluating tests. 
Such knowledge is of importance in making decisions as to 
which modol and which statistic should be usea in such 
evaluations . 



Rasch M cdo) ■ - 

Work has just begun in describing the characteristics 
of data that pruiuce good fit to the Rasch model. In her 
dissertation Panchapakesan (1969) has explored the effect of 
varying item discrimination, tho presence of "bad" items, and 
the effect of guessing on the model. She found that inequality 
of item discrimination, lack of unidimens tonality , and variation 
in guessing decreased fit to the model. The statistic used in 
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evaluating fit to the Rasch model is the chi-square test for 
goodness of fit between observod data and expected values of 
that data. Per a clear presentation of the Rasch model and 
this test of goodness of fit see Wright and PanchapaResan 




( 1969 ), 

In asking the question, “WhAt do data with perfect fit 
to the Rasch model look like?", characteristics of both the 
cLmvcal and Guttman models were considered. Similarity was 
noted between the Guttman and Rasch models. While the Rasch 
model scales Items on easiness and subjects on ability, the 
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Guttman model orders the items on difficulty and the 
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subjects on total scoro. 

T’ is sin,ilarit >’ a ° es Continue, however. Tho Rasch 
model alone determines estimates of subject ability and item 
easiness that have ratio scale characteristics, i.e. the 
ability scale has a taro point that means tori ability ami 
b«l*p Poi'.t on t>e item easiness scale means ,.o easiness or 
infinue difficuliy. Pov the Rasch modelltiie ability ttimates 
are independent of the iter easinesses and the number of items., 

This :s not tie case with i'.uttnan scalin* where scores arc 
the total nunior of correct, responses and thus dependent 
upon. both the number and diiilculty of items. The Rasch 
' Ul '" ! Pfc'iv.urt also produces ability estimates independent 
•>f the sample of subjects used tc scale or calibrate the 
item easinesses. This independence has been well illustrated 
>y KTipht uses). The Gu timer, model docs not attempt to rake 
assumptions mere restrictive ihan that the data mo ordinal on 
(he two diversions • i.e. iter, difficulty and total scoi'c. 
conversely, Ht sea's ncdel is a latent trait model, a model 
thM estimate.* the persons underlying trait while the Gutman 
rnou.l only products n scoro relative to'the items used. 

U ‘' " f ‘ u I'anchapalesan (1565) i* that e<,ual 

item Usc.Uinati on is necessity for Rood lit to the Ratch model. 
Pietro l represents the Horn ehatacteristlc curves (ICC) for 
i 1U-.S over a range of abiUty. These cur/es reptesert the 
probability „f responding correctly to an item if 6 l,*n the 
nbject ability, Fe. ,.<,uni lias dlscriminmeft. these turves 
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need to be parallel. This is not necessary for Guttman 
' scaling. Instead Guttman scaling requires that these 
curves be discrete or nonoverlapping on the range of 
ability measured by each item (Figure 2). 
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Simulation I: Perfect Guttman Scale 

In that the Rasch model is a function of two independent 
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factors, the model is relatively complex add the influences 
of various factors that may enter the model are difficult 
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to determine. For this reason, at the 1969 AERA pre-session 
on Rasch scaling, Wright suggested using simulated data. 

The present study followed this suggestion. This also 
provided a way to control for unwanted variables and to 
"provide a way to compare the two models. In submitting 
simulated perfect Guttman data to the Rasch analysis a 

* . * ' ■ < . . ■ -7 , . 

problem developed. In that items either answered correctly 
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or missed by all subjects are beyond the range of the 
calibration sample the easiness of these items cannot be 
determined. Wh»n this occurs the items are deleted. 
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Similarly, when a subject misses all items or ansvsrs all 
items correctly the ability of these subjects axis': somewhere 

beyond the range of ability measured by this sat os* items 
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and thus cannot be determined. Again these subjects are 

deleted from the response matrix. With perfect Guttman data, 
either at least one item or at least one subject is deleted 
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from the response matrix, leaving another subject or Item 

to be deleted on the next round of truncation. The result 
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is no data to be analysed by the Rasch model. 
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Simulation Il i Near Perfect Guttman' Data and Random- Data 
. To avoid this problem , near perfect Guttman data were 
generated with enough random deviation from a perfect f 
Guttman scale so that few or A o items or subject!* would ■ 
be deleted. This was done by introducing a normally 
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distributed random error variable into the model., The * 
effect of increasing the standard deviation of this error 
was to increase the probability (from sefo to one half) of 
a subject missing an item below his ability level and, 
similarly, to increase the probability oi passing an item 
above his ability level. In that this error was normally 
distributed this probability diminished as. the distance 
of *ho easiness of the items from an item measuring the . 
subjects ability level increased. Kith the error standard 
deviation being small the data would. appear to be near a 
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perfect Guttman scale.' In this case, the data produced a 
fit to the Reach model with a probability of one. 

t ' • 

The nature of the probabilistic model noeds to be 

• "■$ fr . % 

examined here. ■ In that the Rasch model is a probabilistic 
model, distributions of probabilities of item and person 
parameters are assumed and produced. If thtra is no 
variability in the observed acorea from the expected scores 
then the expected probabilities of the model are not met. 
thus this probabilistic model la not appropriate for the data. 
This is whit occurs whan the chi* square probability of fit 
reaches one. This maybe letter seen with sn example. 

« * '* a :• 7: *■ V..’ 



i,lt six coins were repeatedly tossed you would 



not 



expect 



to get exactly three heads and three tails on each toss. 
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If this were the case the probability distribution of these 
events would not be met. 
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in the cose of a perfect Guttman scale no. variability 
is allowed. This is seen in the definition of reproducibility 
Reproducibility is the necessary condition for a perfect 

Guttnan scale^ where from a person's total score his 

; ik 

response pattern to the set of items can be exactly deternined 
In this sense the Guttnan model is a deterministic model 
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contradicting the assumptions of the Rasch model. The 
sought after probability for tho chi-square test of goodness 
of fit is one half. Great deviation either way from one 
half represents a lack of fit to the Rasch model. 

Data sets were then generated with increases in error 
deviation fror» a perfect Guttnan scale. Pit to the Rasch 
model rapidly decreased to Around a probability of .S (Tabli 1) 
To explore this further, 20 sets of completely random data 
wore generate.! and subjected to the Rasch analysis. These 
data ?ets represented the responses of 1000 Ss to 64 items. 

The average probability of goodness of* fit was .414 with 
little variation, S.D. * .178, not significantly different^ 

, »t • . *; ■ ‘u ■ ■ ■ .*■ 

fror a probability of fit of .$00. 

This finding was at first unexpected •» what do such 
date aean? Wright’s Law school Admission Test study (Wright, 



1968) used data that had zero fit to the model which seems 
to be the more common occurrence with real data. In 
examining the original assumptions of the Rasch model this 
persi stance of good fit for random data ct T be explained. ^ 

Dis cussion H -'-y 

Rasch bases his model on three assumptions: 

"1. To each situation of a suhject (v) having to solve 
a test item (i ) , there corresponds a probability of a correct 

■ "ir"- ■ .< fc ■; '.r • .... ./ ' r t': . 
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answer which we will write in the form p • 

Ki**’ 

2. The situational parameter is the product of two 

factors x yi ■ JS v e l (c pertaining to the subject, to 
the item)*', > 

These parameters have been translated to 2 and B 
respectively by Wright (19$$) . 

"3. All answers, given the parameters, are stochastically 
independent." Rasch (1966). f ' 

This term stochastically independent can be translated 
to "local independence." Though neither Rasch nor Wright 
uses this leva in his writing it was used at the 1969 AERA 
pre-session on Rasch scaling. The term local independence 
is also used to describe a basic assumption of Birnbaum's 
model (f'anchsp&kesan, 1969: Lord and Novick 1968). 

Birnbaum's model is similar to Rasch's model but with the 
additional parameter of item discrimination. Tho assumption 

•* • '" ;; f ■'■"‘.v- ; :: iV . 

of local independence provides that "at a fixed point X 
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the probabilities for joint occurrence are products of the 
sapernte probabilities (Lazarsfeld, 1060, p. 85)." In 
other words, "those examined at a given ability level who 
answer a given item correctly are no wore likely to answer 



other Items correctly than are those examinees at the same 
ability level who answer tho given item incorrectly (Lord, 
1966, p. 25)." I.ar 2 arsfeid (1969, p. 497) also said that 



"if a class of poople arc alike in an underlying property 
then the indicators of this property should not be statistically 
related in this class." 

This van explain why completely random data show good 
fit to tho model.' It represents the case of "local" data, 
l.e. of subjects with the same ability level and all items 
at tho difficulty level Measuring that ability. In this case 
the item characteristic curve for each item would coincide and 
thus not meet the clrtorion of a perfect Guttman scale. 

This illustrates another advantage of the Reach model over 
the Guttman model. The ideal and most precise estimates of a 
person's ability are made when the easiness of the items match 

the subjects ability, i.e. with repeated measurement of the 

« '* 4 ■ r : ' 

persons 1 ability. With the Guttman model • person's ability 
is only measured by a single item or within an interval of only 
two items. The precision of measurement is determined hy how 
fine of a discrimination can be made between these two Items. It 
the iters exactly meet his ability level then you would expect 
a 50*50 chance of the person passing each item. In this case the 
perfectness of the Guttman scale would be lacking. 



Simulation III : Distribution of Total Score 

Anotner factor that can be varied in simulating data is the 



distribution of total scores of the subjects.- In that Guttman 
scaling is an ordinal procedure this should have no effect on 
reproducibility. Similarly, in that the Rasch model makes no 
essumpitons about the nature of the distribution of total scores, 



variation in the form of the distribution should havo no effect 
on goodness of fit. Normal distributions varying as to standard 






deviation and uniform distributions varying as to range were 
examined. As expected, no systematic* differences in fit as well 
as no differences in values of the ability estimates were found. 
This last point again illustrates the independence of ability 
estimatos from the item easiness parameters and sample of subjects 
used to calibrate the ability scale. 

Simulation IV: Two Factor Dat a 

Ihe last characteristic to be examined in this paper is 
the effect of combining two sots of Independent data on fit to 
the model. Throe sets of data composed of 1000 subjects 



responding to 32 items were generated with moderate deviation 
from tho perfect Guttman scale. .The distribution of total 
scores was uniform for all three seta. For two sets, named A 
and B, tha rnage of total scares was 32. Except for error 
fluctuation set C was separated with a range of tero and with a 
mean of‘i6. Both data sets A and B had* a chi-square probability 



of fit equaling 1.00. Although both sets of data were generated 
ftom tho same distribution of total acores the two scores assigned 
to each subject vote random \ thus the total scores representing 



the two factors were independent. The results of combining 
those two data sets were surprising. When sets A and B were 
combined the probability for the chi-square test of goodness of ' 

: ' rr:" • •• Vi.- : ' r ' I 1 - IvV--’ ■ ■ v-' ' ' ' -v- - 1 ■■ ... 

fit remained 1.00. When data sets A and C were combined the 
resultant chi-square probability was zero. Data set C by 
itself had a fit of .808. This lack of a fit of probability 
of 1 in data set C was duo to the fact .hat, with truncation, six 
items were eliminated from the data set thus producing a relatively 

. f : * *• • .. • •• :3* "• ••• : . ■* J' *■ ** •<*' j $•&/■■■£ fa ; S.. A-' •• . :?s. • • 

greater amount of deviation from the perfect Tuttman scale. 
Di scussion : 

In an attempt to explain those rosults several sections 
of Wright and Panchapakesrm's computer program for the Rasch 
analysis (1968) wore examined. One of the first matrices 
examined was the score group by item matrix. Elements of 
tills matrix represent the number of times each item was 
answered correctly for each group of subjects receiving 
the same total score. When ordered as to difficutly, 
items from each set alternated. In the case of combining 
sets A and B the effect was to produce two items of 
similar discrimination and easiness when separately there 
had only been one for each set. ‘Thus the only change in 
the nature of the data was the addition of Items to the set. 

The now distribution of score groups or total scores was 
to longer uniform but approached a normal distribution with 
a range of possible scores from 2 to 64. Prom the central 
Unit theorm, if an infinite number of these samples has 
been added, rather than just two, the distribution of total 
scores would have been normal (Wilks, 1962). But in that 
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this model iJ independent of the distribution of total 
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stores this would have had no effect. The effect of 
differences between total score for the two subsets of 
items for each subject was lost. In examining item 
! discrimination each item was similar (Fig. 5); again 
not contradicting the Rasch model. The question still 
remains: M What does this characteristic of the Rasch model 
mean when two independent sots of items combined yield a 
perfect fit?" , . 

The second combination of sets of items, i.e. sets 
A and C, produced a zero fit. Sets A and B were composed 
of items with approximately equal discrimination; the 
i discrimination of the items of set C were poorer. This 
was seen from examining the slopes of the ICC for each 
item, Thus when sets A and C were combined the new set 
had iteirs of varying discrimination and thereby contradicted 
the Rasch model (Figure «}) . 

As was illustrated in this last simulation the 
independence of total scores is alone not enough to reduce 
fit to the model, again the important factor was item 
discrimination. 

Conclusion: 



' J * 









- - y-y:. 

The Rasch model and its chi-square test for goodness of 
fit is highly sensitive to changes in item discrimination/* 
although the slope of the ICC as a measure of item discrimination 
in itself has no effect on goodness of fit. Variations in 
the distribution of total scores has no effect on goodness 
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of fit. Pinaliy in considering two factor data, the 
meaning of the dimensionality of a data set seems 
to mean something different from that of classical test 
theory. What it means needs further study and explanation. 
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Item Characteristic Curves 




Perfect Rasch Scale 




ability 
Figure 3 

Data sets A and B combined, 
(only 6 items illustrated) 
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Figure 4 

Data sets A and C combined, 
(only 10 items illustrated.) 



Table 1 : Mean* Chi-squaro Probabilities for 

Eleven Different Degrees of Error Deviation 
fropi the Perfect Gut t man Seale, 



Error Deviation 
8 

16 

24 

■£;, 32 

40 

4S 

56 

64 

128 

160 

320 

Random Data** 



Mean Chi-square Prob. 
. 986 
0.864 
.691 
.581 
.628 
.427 
.428 
.580 
.582 
.480 
r .565 

.414 



* Eight data sets were generated at each level of error 

deviat ion , these means differed at a .001 level of significance 
*■* This mean was based on 20 sets of Random Data. 
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